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ABSTRACT 



The progress school districts throughout New Jersey made in 
implementing a system of portfolio assessment and the factors that promoted 
or prohibited the implementation of this system were studied. In all, 262 
teachers and 109 administrators completed and returned questionnaires on 
portfolio assessment. Findings indicate that teachers and administrators 
believe that portfolio assessment can provide an effective means of assessing 
students' performance in schools, but that actual practices in their schools 
have not been as successful. Critical to the success of implementing a system 
of portfolio assessment is the need to identify clearly when and how 
portfolios are to be used. It is also important to identify the information 
that portfolios will contain, and teachers and administrators thought that 
these issues had not been resolved. Survey results highlight the importance 
of adequate staff training for teachers and administrators involved in the 
implementation process. (Contains 17 references.) (SLD) 
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Assessment of students’ progress in school has always played an important role in 
education. Teachers administer tests to their students in order to monitor their progress in 
class and to provide feedback for all levels of the educational hierarchy. Teachers use 
these test results to help plan their daily lessons, to set standards of performance in their 
classrooms, to provide diagnostic information about the students they work with, to 
motivate students performance in class, and to evaluate their instructional effectiveness 
(Herman, Aschbacher, & Winters, 1992; King-Shaver, 1993). Students and their parents 
use test results to provide a measure of individual progress towards mastery of skills 
taught: Administrators use test results to provide information about teacher and program 
effectiveness, to identify strengths and weaknesses in the overall curriculum, to identify 
potential staff development needs, and to assist with resource allocation. School districts 
use test data to determine curricular and program needs, to support budget requests, and 
to build public confidence by demonstrating that tax dollars spent are providing positive 
results (Farr, 1992). States use these same test results, at times, for promotion, and 
graduation requirements, for comparison purposes, and for district/teacher accountability. 
Finally, the public looks at these results and makes judgements about the overall 
effectiveness of their schools in comparison with other schools within the same 
community and between surrounding districts. 

During the past twenty years, assessment has taken on a more prominent role as 
public confidence in our educational system has declined. Student performance and 
accountability, two central issues of educational reform movement that began in the 
1970’s have been closely linked with assessment strategies used by school districts. 
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Tests have been used extensively as a way to measure both teacher and student 
accountability and to evaluate the progress of reform efforts instituted at the local, state, 
and national levels. By 1988, 47 states had adopted some form of minimum competency 
testing program that required local school districts to assess students’ progress at some 
point or points between grade one and twelve (Hamisch & Mabry, 1993). These tests, 
according to Hamisch and Mabry (1993) changed the rationale for testing because 
students’ results were being used to shape the curriculum and instructional strategies in 
schools instead of just measuring if students had mastered what was taught. 

Studies commissioned by the United States Department of Education such as A 
Nation at Risk: The Imperative of Education Reform (National Commission on 
Excellence in Education, 1983) and the Carnegie Corporation’s report, (1986), A Nation 
Prepared: Teachers for the Twentv-first Century , began to call for a shift from mastery of 
minimum competencies to promotion of excellence in education through the use of 
standards for students’ performance and teacher accountability. The National Council on 
Education Standards and Testing was commissioned by Congress to recommend long- 
term policies and procedures for implementing national goals outlined in America 2000: 
An Education Strategy , as well as to identify a meaningful way to measure progress 
toward their achievement. The National Council on Education Standards and Testing 
(1993) recommended the development of an assessment system that would be used by 
school districts to provide information to: 

1 . exemplify for students, parents, and teachers the kinds and levels of achievement that 
should be expected; 
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2. improve classroom instruction and the learning outcomes of all students; 

3. inform students, parents, and teachers about students’ progress towards the national 
goals; and 

4. measure and hold students, schools, districts, states, and the nation accountable for 
educational performance 

This call for a shift from assessing minimum competencies to identification of 
essential outcomes and grade-level indicators of progress towards those outcomes led to a 
demand for a new system of assessment. This new system, according to Taylor (1994), 
will “require students to engage in complex tasks using thinking and problem-solving 
skills rather than simply to demonstrate discreet knowledge and skill in applying 
knowledge” (p.232). According to Flood and Lapp (1989), Matthews (1990), and Moses 
(1992), it is important to go beyond the surface features measured by traditional tests and 
analyze the intellectual demands being made on students, the meaningfulness of the tasks, 
the quality of the content being used; and the transfer of these tasks to other skills and 
subjects. Assessment, according to the National Center for Fair and Open Testing 
(1992), should be authentic, measuring actual student performance on written products, 
experiments, exhibitions, and performances. Wiggins (1993) posits that assessment 
strategies must involve students in actual challenges, standards and habits needed for 
success in the workplace and real-life situations where teachers can observe how students 
use evidence, arrange arguments, and take action to address pre-determined problems 
The movement away from strictly using paper and pencil tests to measure what 
students are achieving had led to the ruse in the use of alternative strategies of 
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assessment. The National Council on Testing and Public Policy (1990) in its report. 

From Gate Keeper to Gateway: Transforming Testing in America , urged school districts 
to begin to use alternative forms of assessment to measure student performance in 
schools instead of just relying on standardized, multiple-choice examinations. One 
alternative form, portfolio assessment, rose to prominence in the 1990’s as a means to 
provide teachers and parents with information about what students know and can do. 

Arter (1992) describes portfolio assessment as a form of alternative assessment 
that integrates both authentic and performance strategies for the purpose of demonstrating 
student progress and development over time. Arter and Spandel (1992) describe a 
portfolio as “a purposeful collection of student work that tells the story of the students’ 
efforts, progress and achievement” (p.36). Paulson and Paulson (1991) define a portfolio 
as a carefully crafted portrayal of what a student knows or can do - a purposeful 
integrated collection of student work that demonstrates a student’s effort, progress, or 
achievement in one or more areas. 

When I began my research for my dissertation in 1995, the portfolio assessment 
movement had gained widespread use across the nation - with states such as Vermont and 
Kentucky adopting its use as an integral part of their overall assessment program. 

Schools and districts in New Jersey had been slow to respond to this movement however. 
The purpose of my study was to determine what progress districts throughout New Jersey 
had made to implement a system of portfolio assessment and to identify any factors that 
promoted and/or prohibited its implementation. The study used a survey instrument that 
addressed the following questions: 
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Are portfolios being used in school districts across the state, and with what 
content areas are they being used? 

Are portfolios being used as the sole means to evaluate students and to make 
educational decisions about students, programs, and placement? 

Who participates in the development of the portfolio(s), and what types of data 
are included within them? 

Are issues such as validity and reliability considered in the development of 
scoring procedures? 

How are portfolios scored, and who does the scoring? 

Has adequate training been provided for the staff involved, and what kinds of 
training have been provided? 

What has been the reaction of teachers and students involved in using portfolios? 

What factors, if any, have prohibited implementation of portfolio assessment? 

Is there a difference regarding teachers’ and administrators’ opinions about the 
use of portfolio assessment? 

Is there a difference between teachers’ and administrators’ beliefs with what is 
actually occurring in districts regarding implementation of this assessment strategy? 

METHOD 

The study compared the opinions of teachers and administrators in K-8 school 
districts throughout New Jersey regarding their beliefs about use of portfolio assessment 
with what was actually occurring in their districts. Participants in the study included 
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teachers and administrators from 94 school districts in the State. In all, 262 teachers and 
109 administrators completed and returned survey instruments. 

The districts that participated in the study represented all geographic regions of 
the State and reflected a mix of urban, suburban, and rural districts. Suburban districts, 
however, were the predominant type districts that volunteered to participate in the study. 

Two versions of a survey instrument, one for teachers’ use and another for 
adminis trators were developed to determine the status of the use of portfolios to assess 
students’ progress and to determine if teachers and administrators shared common beliefs 
about fheir use. Both surveys contained similar questions where respondents were asked 
to answer questions by selecting yes, no, or no opinion. Narrative questions were also 
included where teachers and administrators could elaborate about the successes and/or 
problems they had experienced as a result of using portfolios. In addition, the 
administrative survey included nine additional questions regarding evaluative information 
that used a Likert scale to elicit opinions from Administrators responding to the surveys. 
A final section on both survey instruments asked the respondents six demographic 
questions about themselves and their school. A pilot study was conducted using school 
districts in Pennsylvania prior to their final use. 

STATISTICAL METHODS 

Frequency tables were used for the first twelve questions of both survey 
instruments. These tables provided percentages of responses for both teachers and 
administrators regarding actual beliefs held versus practices occurring in their districts. 

A Mann - Whitney U Test, a non-parametric test, was used to compare the mean rank 



O 

ERLC 



8 



7 



order of teacher/administrator responses to determine if a discrepancy existed between 
the beliefs of teachers and administrators versus actual practice. Another non-parametric 
test, Pearson’s Chi-Square Analysis, was used to test if a significant difference existed 
between teachers and administrators concerning their responses. This test identified if a 
contingence exists when you compare beliefs versus practiced responses. A significance 
level of .05 was established for both non-parametric tests. 

Data obtained from both survey instruments was primarily categorical and 
provided binomial distributions for responses for each item. Since questions asked on 
both surveys would be analyzed independently, it was not necessary to determine if they 
were measuring the same concept. A reliability analysis was used, however, for the 
Likert-type questions asked in the administrative survey. A correlation matrix and a z- 
test were used to determine if the questions were homogeneous and measured the same 
concept. An alpha score of .91 was obtained for reaction questions indicating high 
reliability, but the alpha score for factor questions was only .69. 

PRESENTATION OF FINDINGS 

In order to effectively present the data obtained from the two questionnaires, the 
research questions were used in conjunction with specific questions posed in each survey. 
Research Question 1: Are portfolios being used in school districts and what content 
areas are they being used in? 

Results from both survey instruments indicate that teachers and administrators 
believe that using portfolios is an effective way to assess students in all grades in 
elementary school. Ninety-three percent of the teachers and 99% of the administrators 
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indicate that the most appropriate grade span for the use of portfolios is kindergarten 
through second grade. A comparison of these two data sources was significant. The 
difference, however, between teachers’ beliefs versus their actual practices in their 
schools at all grade levels was also significant. This indicates that actual implementation 
of portfolio assessment was not as successful as teachers had anticipated. 

Administrators’ responses comparing beliefs and practices were also significantly 
different for the use of portfolios to assess students at both grades 3-5 and 6-8. 

Responses on the open-ended question tend to support this finding that administrators 
reported positive reactions regarding implementation of portfolios more frequently than 
did teachers. 

The subjects most often identified as being most appropriate for use of portfolios 
(over 90% of teachers and administrators) were reading and language arts. Other subjects 
noted by administrators include mathematics, art, and social studies. The discrepancy 
between actual implementation and teachers’ beliefs regarding all subjects with the 
exception of foreign language was significant. A discrepancy between administrators’ 
beliefs and actual implementation was identified regarding mathematics and social 
studies. Overall, teachers and administrators tend to agree that implementation of 
portfolios in all content areas, with the exception of language arts, was not as successful. 
Research Question 2: Are portfolios being used as the sole means to make educational 
decisions about students’ programs and placement? 

Data collected from both survey instruments indicate that teachers and 
administrators believe that portfolios should be used in conjunction with other assessment 
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measures. In fact, 86% of all teachers and 93% of administrators responding to the 
surveys indicated this. Teachers (95%) and administrators (97%) believe that use of 
portfolios is very effective when conducting parent conferences. Actual practices in 
school districts were markedly different with the discrepancy between beliefs and 
practices for administrators regarding portfolio use in conjunction with other assessment 
measures being significant. 

Over 90% of the teachers surveyed believed that portfolios would be helpful to 
make decisions regarding students’ instructional programs and academic placement. 
Administrators’ results were similar, but tended to be less favorable with the exception of 
use of portfolios to make decisions regarding instructional practices. When comparing 
both groups’ beliefs with actual implementation, significant discrepancies were 
identified. Both teachers and administrators agree that portfolios were not being used to 
make educational decisions as they had anticipated. The discrepancy between teachers 
and administrators regarding actual instructional practices was also significant at the .05 
level. 

Research Question 3: Who participates in the development of the portfolios and what 
types of data are included within them? 

Eighty-six percent of the teachers indicated that students should be involved in 
selecting work samples to include within the portfolios while 9% believed that they, 
alone, should select what material to include. Ninety-four percent of the administrators 
also believed that selecting pieces to include within portfolios should’ be a joint 
responsibility between teachers and students. A discrepancy between beliefs and 
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practices regarding this issue was significant at the .95 level of confidence for teachers’ 
responses to both questions and administrators’ responses regarding teachers, alone, 
selecting pieces to include within the portfolios. This indicates that selection of students’ 
work to include in the portfolios was actually more teacher-directed than the beliefs 
expressed by the teachers and administrators. 

Teachers indicated that portfolios should include a variety of information 
including informal evaluations (87%), checklists (90%), audio/visual tapes (93%), 
student-selected pieces (93%), writing assignments (96%) and individual projects (92%). 
Eighty percent believed that teacher-made tests should be included, while 54% thought 
that standardized tests should be used. Administrators’ beliefs were similar to teachers 
for all categories of information. In practice, teachers (75%) and administrators (85%) 
indicated that writing assignments were most often included within portfolios. This 
supports previous finding that language arts is the content area most frequently utilized 
for portfolio assessment. Differences between beliefs versus practice were significant for 
both teachers and administrators regarding the use of informal evaluations, 
checklists/surveys, writing assignments, individual/group projects, teacher-made tests, 
and standardized tests. Use of audio/visual tapes was found to be significantly different 
for teachers and tended to be so for administrators as well. A comparison of beliefs 
versus practice for teachers and administrators identified that a discrepancy exists 
regarding use of writing assignments and individual/group projects. Implications of these 
results indicate that data included within portfolios tended to be more test related and 
conflicted with beliefs held by both teachers and administrators. Narrative responses for 
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both groups also support this dichotomy in that several teachers and administrators noted 
concerns regarding what types of information to include within the portfolios and how to 
assemble these data into meaningful collections of students’ works. 

Research Question 4: Are issues such as validity and reliability considered in the 
development of scoring procedures? 

This question was only addressed on the administrative survey. Only 23% of the 
administrators indicated that issues of reliability and validity have been considered in the 
development of scoring procedures. This was supported by narrative comments made by 
both teachers and administrators alike. Teachers frequently noted concerns regarding 
consistency of implementation of portfolio assessment, the need to establish clear scoring 
rubrics, and problems associated with two raters reaching the same conclusions about a 
student’s work. Administrators’ comments were more direct, specifically citing the need 
to establish both validity and reliability through uniform implementation across the 
grades. Administrators also noted concerns related to independent observer agreement 
and the need for clearly defined rubrics. 

Research Question 5: How are portfolios scored and who does the scoring? 

Teachers (77%) and administrators (89%) indicated that clear and identifiable 
criteria should be established to judge portfolios and that they should be scored using a 
holistic approach. Only 42% of the teachers and 52% of the administrators believed that 
the portfolios should be scored by a small group of well-trained and evaluated raters. 
When comparing actual practices in districts, significant differences were identified for 
teachers regarding all three issues. Administrators only differed concerning scoring of 
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portfolios by raters and use of a holistic approach. Implementation of assessment 
strategies used to evaluate portfolios in schools participating in the study was 
significantly different in comparison to the beliefs of survey participants. When 
comparing only beliefs of teachers and administrators, the issue of having clear and 
identifiable criteria to judge portfolios was also significant. This difference was not 
significant, however, when actual practices were compared for both groups. Narrative 
comments made by both teachers and administrators described problems related to 
consistency in scoring, the need to establish clearly defined rubrics, and the importance 
of additional training for staff members to use holistic scoring. 

Research Question 6: Has adequate training been provided for the staff involved, and 
what kinds of training have been provided? 

Only 24% of the teachers and 28% of the administrators believed that adequate 
training has been provided in their districts. Administrators also indicated that the lack of 
adequate staff development was one of the factors prohibiting implementation of 
portfolio assessment in their school. 

Teachers and administrators tend to disagree on the types of training that have 
been offered. Teachers’ responses were less favorable regarding all types of training 
including hands-on programs, series of training programs, out-of-district programs and 
follow-up tr ainin g. Significant differences were found when comparing practices in 
districts identified by teachers and administrators for all types of training offered in their 
schools. Narrative comments from teachers clearly indicated their concern over lack of 
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training to implement this program and its need to ensure that staff involved can 
universally implement portfolio assessment strategies. 

Administrators’ comments also identified concerns regarding training but focused 
on costs and time involved to effectively train the staff. Practices regarding types of 
training provided were higher for administrators with out-of-district programs being 
significant at the .001 level. Administrators (31%) did note that follow-up training was 
provided on a limited basis - an element clearly essential to the successful 
implementation of any new program. 

Research Question 7: What is the reaction of teachers and students using portfolios? 

Comments made by 171 teachers who responded to the narrative questions 
included both positive and negative reactions. Teachers indicated that portfolios are good 
tools to use when assessing students and that they provide valuable information about 
students’ progress and mastery of skills across the curriculum. Many teachers noted that 
portfolios were a wonderful way to evaluate the strengths and weaknesses of their 
students - providing a complete picture of each child’s educational growth and 
development. Teachers noted that using portfolios improved their knowledge and 
understanding of how children learn and increased collegial interaction between their 
peers. Teachers repeatedly noted improved students’ self-esteem because students could 
clearly see success and improvement in their work. They also noted that students were 
becoming active participants in the learning process - selecting and critiquing their own 
work. Finally, several teachers cited positive parent feedback for the program - 
especially during parent-teacher conferences. 




15 



14 



Administrators’ reactions regarding teachers and students were based upon 
Likert-type questions incorporated within their questionnaires and a summary of the 
narrative comments they made. Administrators tended to agree that use of portfolio 
assessment has encouraged teachers to change their instructional practices and that it is a 
powerful way to link curriculum and instruction with assessment. Administrators also 
agreed that student reaction to the use of portfolios was positive. Results on questions 
regarding students accepting constructive criticism, the overall success of the program, 
and plans to expand portfolio assessment use tended to indicate agreement but to a lesser 
degree. Narrative comments made by administrators support these reactions and 
complement positives noted by teachers. Positive reactions for the teachers include better 
understanding of the teaching/leaming process by staff involved and positive changes in 
the instructional practices of teachers. They also noted improved students’ self-esteem, 
better oral and written communication skills, and increased support from the parents and 
the community. Several administrators noted that portfolio assessment was a more 
accurate way to assess students, and that this process actively involved the students. 
Research Question 8: What factors, if any, have prohibited implementation of portfolio 
assessment? 

Clearly, the teachers identified several concerns regarding portfolio assessment 
that have impacted its successful implementation. The issue of time - to organize 
portfolios and keep them current, to meet with children to pick selections and reflect on 
their work, and to score portfolios - was most frequently noted as a major concern. As 
previously described, teachers cited difficulty with knowing what information to include 
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within the portfolios and how to interpret and evaluate the work included. Many teachers 
believed that the process was too subjective and lacked consistency; resulting in 
duplication of paperwork and limited commitment on the part of all teachers involved. 
Other concerns noted include a lack of administrative support, the need for additional 
staff development training and a lack of parental support because many parents were not 
comfortable with an assessment system that is not based on test results and grades. 
Problems with managing and storing the portfolios and students who are often reluctant 
to accept greater responsibilities for their own work were also identified as problems. 

Factors identified by administrators that have prohibited successful 
implementation of portfolio assessment on the Likert-type questions incorporated within 
their questionnaire include the financial costs involved, the lack of adequate staff 
training, time to implement the program, and the additional responsibilities involved. 
Administrators also identified an additional concern in their narrative comments that 
focused on the lack of commitment on the part of all staff involved. This lack of 
commitment was noted by administrators on the Likert-type question (mean —2.66) but 
was not as critical as other factors identified. 

Limitations of the Study 

The generalization of this study is limited to the population from which the 
sample was drawn for this study. Since this study was only conducted in K-8 school 
districts in New Jersey, inferences for larger school districts within the state and other 
public schools in other states would be inappropriate. Finally, the study focused on the 
attitudes and opinions of teachers and administrators and not with a collection of 
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objective data, so it is limited to the degree of accuracy and candor of the respondents’ 
replies. 

Implications of the Study 

The findings of this study indicate that teachers and administrators believe that 
portfolio assessment can provide an effective means of assessing students’ performance 
in schools but that actual practices in their schools have not been as successful. Critical 
to the success of implementing a system of portfolio assessment is the need to clearly 
identify when and how portfolios are going to be used. It is also important to identify 
what information will be included within the portfolios, establish clear and identifiable 
rubrics to judge the portfolios, and provide training and support for the staff involved 
with its implementation. Overall, teachers and administrators expressed reluctance 
regarding implementation of portfolio assessment because these issues had not been 
clearly resolved. This reluctance clearly demonstrates the importance of building a base 
of support with teachers and principals before implementing change. It is essential to 
gather enough information and to share it through effective staff development programs. 
In fact, the importance of providing adequate staff training for both teachers and 
administrators involved with implementation of portfolio assessment is essential for the 
program to have any chance of success. This training must be ongoing and focus on all 
aspects of the program. 

Both teachers and administrators cite many positive implications regarding 
implementation of portfolio assessment for staff, students, and parents, which can 
enhance a school’s overall assessment program. The results of this study should be of 
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interest to teachers and administrators planning to utilize portfolios as an integral 
component of an assessment model. 
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